30.0 KB · ⇊ 19 parts · ⤓ Micron 30.0 KB · ⤓ HTML 227.4 KB

top

In mathematics, the operator norm measures the "size" of certain linear operators by assigning each a real number called its operator norm. Formally, it is a norm defined on the space of bounded linear operators between two given normed vector spaces. Informally, the operator norm ‖ ‖ T ‖ ‖ {\displaystyle \|T\|} of a linear map T : X → → Y {\displaystyle T:X\to Y} is the maximum factor by which it "lengthens" vectors.

• Introduction and definition

• Examples

• Equivalent definitions

• Properties

• Table of common operator norms

• Operators on a Hilbert space

• See also

• Notes

• References

Given two normed vector spaces V {\displaystyle V} and W {\displaystyle W} (over the same base field, either the real numbers R {\displaystyle \mathbb {R} } or the complex numbers C {\displaystyle \mathbb {C} } ), a linear map A : V → → W {\displaystyle A:V\to W} is continuous if and only if there exists a real number c {\displaystyle c} such thatcite-ref-1[1] ‖ ‖ A v ‖ ‖ ≤ ≤ c ‖ ‖ v ‖ ‖ for all v ∈ ∈ V . {\displaystyle \|Av\|\leq c\|v\|\quad {\text{ for all }}v\in V.}

The norm on the left is the one in W {\displaystyle W} and the norm on the right is the one in V {\displaystyle V} . Intuitively, the continuous operator A {\displaystyle A} never increases the length of any vector by more than a factor of c . {\displaystyle c.} Thus the image of a bounded set under a continuous operator is also bounded. Because of this property, the continuous linear operators are also known as bounded operators. In order to "measure the size" of A , {\displaystyle A,} one can take the infimum of the numbers c {\displaystyle c} such that the above inequality holds for all v ∈ ∈ V . {\displaystyle v\in V.} This number represents the maximum scalar factor by which A {\displaystyle A} "lengthens" vectors. In other words, the "size" of A {\displaystyle A} is measured by how much it "lengthens" vectors in the "biggest" case. So we define the operator norm of A {\displaystyle A} as ‖ ‖ A ‖ ‖ op = inf { c ≥ ≥ 0 : ‖ ‖ A v ‖ ‖ ≤ ≤ c ‖ ‖ v ‖ ‖ for all v ∈ ∈ V } . {\displaystyle \|A\|_{\text{op}}=\inf\{c\geq 0:\|Av\|\leq c\|v\|{\text{ for all }}v\in V\}.}

The infimum is attained as the set of all such c {\displaystyle c} is closed, nonempty, and bounded from below.cite-ref-2[2]

It is important to bear in mind that this operator norm depends on the choice of norms for the normed vector spaces V {\displaystyle V} and W {\displaystyle W} .

Every real m {\displaystyle m} -by- n {\displaystyle n} matrix corresponds to a linear map from R n {\displaystyle \mathbb {R} ^{n}} to R m . {\displaystyle \mathbb {R} ^{m}.} Each pair of the plethora of (vector) norms applicable to real vector spaces induces an operator norm for all m {\displaystyle m} -by- n {\displaystyle n} matrices of real numbers; these induced norms form a subset of matrix norms.

If we specifically choose the Euclidean norm on both R n {\displaystyle \mathbb {R} ^{n}} and R m , {\displaystyle \mathbb {R} ^{m},} then the matrix norm given to a matrix A {\displaystyle A} is the square root of the largest eigenvalue of the matrix A ∗ ∗ A {\displaystyle A^{*}A} (where A ∗ ∗ {\displaystyle A^{*}} denotes the conjugate transpose of A {\displaystyle A} ).cite-ref-3[3] This is equivalent to assigning the largest singular value of A . {\displaystyle A.}

Passing to a typical infinite-dimensional example, consider the sequence space ℓ ℓ 2 , {\displaystyle \ell ^{2},} which is an Lp space, defined by ℓ ℓ 2 = { ( a n ) n ≥ ≥ 1 : a n ∈ ∈ C , ∑ ∑ n | a n | 2 < ∞ ∞ } . {\displaystyle \ell ^{2}=\left\{(a_{n})_{n\geq 1}:\;a_{n}\in \mathbb {C} ,\;\sum _{n}|a_{n}|^{2}<\infty \right\}.}

This can be viewed as an infinite-dimensional analogue of the Euclidean space C n . {\displaystyle \mathbb {C} ^{n}.} Now consider a bounded sequence s ∙ ∙ = ( s n ) n = 1 ∞ ∞ . {\displaystyle s_{\bullet }=\left(s_{n}\right)_{n=1}^{\infty }.} The sequence s ∙ ∙ {\displaystyle s_{\bullet }} is an element of the space ℓ ℓ ∞ ∞ , {\displaystyle \ell ^{\infty },} with a norm given by ‖ s ∙ ∙ ‖ ∞ ∞ = sup n | s n | . {\displaystyle \left\|s_{\bullet }\right\|_{\infty }=\sup _{n}\left|s_{n}\right|.}

Define an operator T s {\displaystyle T_{s}} by pointwise multiplication: ( a n ) n = 1 ∞ ∞ ↦ ↦ T s ( s n ⋅ ⋅ a n ) n = 1 ∞ ∞ . {\displaystyle \left(a_{n}\right)_{n=1}^{\infty }\;{\stackrel {T_{s}}{\mapsto }}\;\ \left(s_{n}\cdot a_{n}\right)_{n=1}^{\infty }.}

The operator T s {\displaystyle T_{s}} is bounded with operator norm ‖ T s ‖ op = ‖ s ∙ ∙ ‖ ∞ ∞ . {\displaystyle \left\|T_{s}\right\|_{\text{op}}=\left\|s_{\bullet }\right\|_{\infty }.}

This discussion extends directly to the case where ℓ ℓ 2 {\displaystyle \ell ^{2}} is replaced by a general L p {\displaystyle L^{p}} space with p > 1 {\displaystyle p>1} and ℓ ℓ ∞ ∞ {\displaystyle \ell ^{\infty }} replaced by L ∞ ∞ . {\displaystyle L^{\infty }.}

Let A : V → → W {\displaystyle A:V\to W} be a linear operator between normed spaces. The first four definitions are always equivalent, and if in addition V ≠ ≠ { 0 } {\displaystyle V\neq \{0\}} then they are all equivalent:

‖ ‖ A ‖ ‖ op = inf { c ≥ ≥ 0 : ‖ ‖ A v ‖ ‖ ≤ ≤ c ‖ ‖ v ‖ ‖ for all v ∈ ∈ V } = sup { ‖ ‖ A v ‖ ‖ : ‖ ‖ v ‖ ‖ ≤ ≤ 1 and v ∈ ∈ V } = sup { ‖ ‖ A v ‖ ‖ : ‖ ‖ v ‖ ‖ < 1 and v ∈ ∈ V } = sup { ‖ ‖ A v ‖ ‖ : ‖ ‖ v ‖ ‖ ∈ ∈ { 0 , 1 } and v ∈ ∈ V } = sup { ‖ ‖ A v ‖ ‖ : ‖ ‖ v ‖ ‖ = 1 and v ∈ ∈ V } this equality holds if and only if V ≠ ≠ { 0 } = sup { ‖ ‖ A v ‖ ‖ ‖ ‖ v ‖ ‖ : v ≠ ≠ 0 and v ∈ ∈ V } this equality holds if and only if V ≠ ≠ { 0 } . {\displaystyle {\begin{alignedat}{4}\|A\|_{\text{op}}&=\inf &&\{c\geq 0~&&:~\|Av\|\leq c\|v\|~&&~{\text{ for all }}~&&v\in V\}\\&=\sup &&\{\|Av\|~&&:~\|v\|\leq 1~&&~{\mbox{ and }}~&&v\in V\}\\&=\sup &&\{\|Av\|~&&:~\|v\|<1~&&~{\mbox{ and }}~&&v\in V\}\\&=\sup &&\{\|Av\|~&&:~\|v\|\in \{0,1\}~&&~{\mbox{ and }}~&&v\in V\}\\&=\sup &&\{\|Av\|~&&:~\|v\|=1~&&~{\mbox{ and }}~&&v\in V\}\;\;\;{\text{ this equality holds if and only if }}V\neq \{0\}\\&=\sup &&{\bigg \{}{\frac {\|Av\|}{\|v\|}}~&&:~v\neq 0~&&~{\mbox{ and }}~&&v\in V{\bigg \}}\;\;\;{\text{ this equality holds if and only if }}V\neq \{0\}.\\\end{alignedat}}}

If V = { 0 } {\displaystyle V=\{0\}} then the sets in the last two rows will be empty, and consequently their supremums over the set [ − − ∞ ∞ , ∞ ∞ ] {\displaystyle [-\infty ,\infty ]} will equal − − ∞ ∞ {\displaystyle -\infty } instead of the correct value of 0. {\displaystyle 0.} If the supremum is taken over the set [ 0 , ∞ ∞ ] {\displaystyle [0,\infty ]} instead, then the supremum of the empty set is 0 {\displaystyle 0} and the formulas hold for any V . {\displaystyle V.}

Importantly, a linear operator A : V → → W {\displaystyle A:V\to W} is not, in general, guaranteed to achieve its norm ‖ ‖ A ‖ ‖ op = sup { ‖ ‖ A v ‖ ‖ : ‖ ‖ v ‖ ‖ ≤ ≤ 1 , v ∈ ∈ V } {\displaystyle \|A\|_{\text{op}}=\sup\{\|Av\|:\|v\|\leq 1,v\in V\}} on the closed unit ball { v ∈ ∈ V : ‖ ‖ v ‖ ‖ ≤ ≤ 1 } , {\displaystyle \{v\in V:\|v\|\leq 1\},} meaning that there might not exist any vector u ∈ ∈ V {\displaystyle u\in V} of norm ‖ ‖ u ‖ ‖ ≤ ≤ 1 {\displaystyle \|u\|\leq 1} such that ‖ ‖ A ‖ ‖ op = ‖ ‖ A u ‖ ‖ {\displaystyle \|A\|_{\text{op}}=\|Au\|} (if such a vector does exist and if A ≠ ≠ 0 , {\displaystyle A\neq 0,} then u {\displaystyle u} would necessarily have unit norm ‖ ‖ u ‖ ‖ = 1 {\displaystyle \|u\|=1} ). R.C. James proved James's theorem in 1964, which states that a Banach space V {\displaystyle V} is reflexive if and only if every bounded linear functional f ∈ ∈ V ∗ ∗ {\displaystyle f\in V^{*}} achieves its norm on the closed unit ball.cite-ref-footnotediestel19846-4-0[4] It follows, in particular, that every non-reflexive Banach space has some bounded linear functional (a type of bounded linear operator) that does not achieve its norm on the closed unit ball.

If A : V → → W {\displaystyle A:V\to W} is bounded thencite-ref-footnoterudin199192-115-5-0[5] ‖ ‖ A ‖ ‖ op = sup { | w ∗ ∗ ( A v ) | : ‖ ‖ v ‖ ‖ ≤ ≤ 1 , ‖ w ∗ ∗ ‖ ≤ ≤ 1 where v ∈ ∈ V , w ∗ ∗ ∈ ∈ W ∗ ∗ } {\displaystyle \|A\|_{\text{op}}=\sup \left\{\left|w^{*}(Av)\right|:\|v\|\leq 1,\left\|w^{*}\right\|\leq 1{\text{ where }}v\in V,w^{*}\in W^{*}\right\}} andcite-ref-footnoterudin199192-115-5-1[5] ‖ ‖ A ‖ ‖ op = ‖ t A ‖ op {\displaystyle \|A\|_{\text{op}}=\left\|{}^{t}A\right\|_{\text{op}}} where t A : W ∗ ∗ → → V ∗ ∗ {\displaystyle {}^{t}A:W^{*}\to V^{*}} is the transpose of A : V → → W , {\displaystyle A:V\to W,} which is the linear operator defined by w ∗ ∗ ↦ ↦ w ∗ ∗ ∘ ∘ A . {\displaystyle w^{*}\,\mapsto \,w^{*}\circ A.}

The operator norm is indeed a norm on the space of all bounded operators between V {\displaystyle V} and W {\displaystyle W} . This means ‖ ‖ A ‖ ‖ op ≥ ≥ 0 and ‖ ‖ A ‖ ‖ op = 0 if and only if A = 0 , {\displaystyle \|A\|_{\text{op}}\geq 0{\mbox{ and }}\|A\|_{\text{op}}=0{\mbox{ if and only if }}A=0,} ‖ ‖ a A ‖ ‖ op = | a | ‖ ‖ A ‖ ‖ op for every scalar a , {\displaystyle \|aA\|_{\text{op}}=|a|\|A\|_{\text{op}}{\mbox{ for every scalar }}a,} ‖ ‖ A + B ‖ ‖ op ≤ ≤ ‖ ‖ A ‖ ‖ op + ‖ ‖ B ‖ ‖ op . {\displaystyle \|A+B\|_{\text{op}}\leq \|A\|_{\text{op}}+\|B\|_{\text{op}}.}

The following inequality is an immediate consequence of the definition: ‖ ‖ A v ‖ ‖ ≤ ≤ ‖ ‖ A ‖ ‖ op ‖ ‖ v ‖ ‖ for every v ∈ ∈ V . {\displaystyle \|Av\|\leq \|A\|_{\text{op}}\|v\|\ {\mbox{ for every }}\ v\in V.}

The operator norm is also compatible with the composition, or multiplication, of operators: if V {\displaystyle V} , W {\displaystyle W} and X {\displaystyle X} are three normed spaces over the same base field, and A : V → → W {\displaystyle A:V\to W} and B : W → → X {\displaystyle B:W\to X} are two bounded operators, then it is a sub-multiplicative norm, that is: ‖ ‖ B A ‖ ‖ op ≤ ≤ ‖ ‖ B ‖ ‖ op ‖ ‖ A ‖ ‖ op . {\displaystyle \|BA\|_{\text{op}}\leq \|B\|_{\text{op}}\|A\|_{\text{op}}.}

For bounded operators on V {\displaystyle V} , this implies that operator multiplication is jointly continuous.

It follows from the definition that if a sequence of operators converges in operator norm, it converges uniformly on bounded sets.

By choosing different norms for the codomain, used in computing ‖ ‖ A v ‖ ‖ {\displaystyle \|Av\|} , and the domain, used in computing ‖ ‖ v ‖ ‖ {\displaystyle \|v\|} , we obtain different values for the operator norm. Some common operator norms are easy to calculate, and others are NP-hard. Except for the NP-hard norms, all these norms can be calculated in N 2 {\displaystyle N^{2}} operations (for an N × × N {\displaystyle N\times N} matrix), with the exception of the ℓ ℓ 2 − − ℓ ℓ 2 {\displaystyle \ell _{2}-\ell _{2}} norm (which requires N 3 {\displaystyle N^{3}} operations for the exact answer, or fewer if you approximate it with the power method or Lanczos iterations).

| | | Co-domain | Co-domain | Co-domain |

|---|---|---|---|---|

| | | ℓ 1 {\displaystyle \ell _{1}} | ℓ 2 {\displaystyle \ell _{2}} | ℓ ∞ {\displaystyle \ell _{\infty }} |

| Domain | ℓ 1 {\displaystyle \ell _{1}} | Maximum ℓ 1 {\displaystyle \ell _{1}} norm of a column | Maximum ℓ 2 {\displaystyle \ell _{2}} norm of a column | Maximum ℓ ∞ {\displaystyle \ell _{\infty }} norm of a column |

| Domain | ℓ 2 {\displaystyle \ell _{2}} | NP-hard | Maximum singular value | Maximum ℓ 2 {\displaystyle \ell _{2}} norm of a row |

| Domain | ℓ ∞ {\displaystyle \ell _{\infty }} | NP-hard | NP-hard | Maximum ℓ 1 {\displaystyle \ell _{1}} norm of a row |

The norm of the adjoint or transpose can be computed as follows. We have that for any p , q , {\displaystyle p,q,} then ‖ ‖ A ‖ ‖ p → → q = ‖ ‖ A ∗ ∗ ‖ ‖ q ′ → → p ′ {\displaystyle \|A\|_{p\rightarrow q}=\|A^{*}\|_{q'\rightarrow p'}} where p ′ , q ′ {\displaystyle p',q'} are Hölder conjugate to p , q , {\displaystyle p,q,} that is, 1 / p + 1 / p ′ = 1 {\displaystyle 1/p+1/p'=1} and 1 / q + 1 / q ′ = 1. {\displaystyle 1/q+1/q'=1.}

Suppose H {\displaystyle H} is a real or complex Hilbert space. If A : H → → H {\displaystyle A:H\to H} is a bounded linear operator, then we have ‖ ‖ A ‖ ‖ op = ‖ A ∗ ∗ ‖ op {\displaystyle \|A\|_{\text{op}}=\left\|A^{*}\right\|_{\text{op}}} and ‖ A ∗ ∗ A ‖ op = ‖ ‖ A ‖ ‖ op 2 , {\displaystyle \left\|A^{*}A\right\|_{\text{op}}=\|A\|_{\text{op}}^{2},} where A ∗ ∗ {\displaystyle A^{*}} denotes the adjoint operator of A {\displaystyle A} (which in Euclidean spaces with the standard inner product corresponds to the conjugate transpose of the matrix A {\displaystyle A} ).

In general, the spectral radius of A {\displaystyle A} is bounded above by the operator norm of A {\displaystyle A} : ρ ρ ( A ) ≤ ≤ ‖ ‖ A ‖ ‖ op . {\displaystyle \rho (A)\leq \|A\|_{\text{op}}.}

To see why equality may not always hold, consider the Jordan canonical form of a matrix in the finite-dimensional case. Because there are non-zero entries on the superdiagonal, equality may be violated. The quasinilpotent operators is one class of such examples. A nonzero quasinilpotent operator A {\displaystyle A} has spectrum { 0 } . {\displaystyle \{0\}.} So ρ ρ ( A ) = 0 {\displaystyle \rho (A)=0} while ‖ ‖ A ‖ ‖ op > 0. {\displaystyle \|A\|_{\text{op}}>0.}

However, when a matrix N {\displaystyle N} is normal, its Jordan canonical form is diagonal (up to unitary equivalence); this is the spectral theorem. In that case it is easy to see that ρ ρ ( N ) = ‖ ‖ N ‖ ‖ op . {\displaystyle \rho (N)=\|N\|_{\text{op}}.}

This formula can sometimes be used to compute the operator norm of a given bounded operator A {\displaystyle A} : define the Hermitian operator B = A ∗ ∗ A , {\displaystyle B=A^{*}A,} determine its spectral radius, and take the square root to obtain the operator norm of A . {\displaystyle A.}

The space of bounded operators on H , {\displaystyle H,} with the topology induced by operator norm, is not separable. For example, consider the Lp space L 2 [ 0 , 1 ] , {\displaystyle L^{2}[0,1],} which is a Hilbert space. For 0 < t ≤ ≤ 1 , {\displaystyle 0<t\leq 1,} let Ω Ω t {\displaystyle \Omega _{t}} be the characteristic function of [ 0 , t ] , {\displaystyle [0,t],} and P t {\displaystyle P_{t}} be the multiplication operator given by Ω Ω t , {\displaystyle \Omega _{t},} that is, P t ( f ) = f ⋅ ⋅ Ω Ω t . {\displaystyle P_{t}(f)=f\cdot \Omega _{t}.}

Then each P t {\displaystyle P_{t}} is a bounded operator with operator norm 1 and ‖ P t − − P s ‖ op = 1 for all t ≠ ≠ s . {\displaystyle \left\|P_{t}-P_{s}\right\|_{\text{op}}=1\quad {\mbox{ for all }}\quad t\neq s.}

But { P t : 0 < t ≤ ≤ 1 } {\displaystyle \{P_{t}:0<t\leq 1\}} is an uncountable set. This implies the space of bounded operators on L 2 ( [ 0 , 1 ] ) {\displaystyle L^{2}([0,1])} is not separable, in operator norm. One can compare this with the fact that the sequence space ℓ ℓ ∞ ∞ {\displaystyle \ell ^{\infty }} is not separable.

The associative algebra of all bounded operators on a Hilbert space, together with the operator norm and the adjoint operation, yields a C*-algebra.

• Banach–Mazur compactum – Concept in functional analysis

• Continuous linear operator – Function between topological vector spaces

• Contraction (operator theory) – Bounded operators with sub-unit norm

• Discontinuous linear map

• Dual norm – Measurement on a normed vector space

• Matrix norm – Norm on a vector space of matrices

• Norm (mathematics) – Length in a vector space

• Normed space – Vector space on which a distance is definedPages displaying short descriptions of redirect targets

• Operator algebra – Branch of functional analysis

• Operator theory – Mathematical field of study

• Topologies on the set of operators on a Hilbert space

• Unbounded operator – Linear operator defined on a dense linear subspace

cite-note-11. ↑ citerefkreyszig1978Kreyszig, Erwin (1978), Introductory functional analysis with applications, John Wiley & Sons, p. 97, ISBN 9971-51-381-1

cite-note-22. ↑ See e.g. Lemma 6.2 of Aliprantis & Border (2007).

cite-note-33. ↑ citerefweissteinWeisstein, Eric W. "Operator Norm". mathworld.wolfram.com. Retrieved 2020-03-14.

cite-note-footnotediestel19846-44. ↑ Diestel 1984, p. 6.

cite-note-footnoterudin199192-115-55. ↑ Rudin 1991, pp. 92–115.

cite-note-61. section 4.3.1, Joel Tropp's PhD thesis, [1]

• citerefaliprantisborder2007Aliprantis, Charalambos D.; Border, Kim C. (2007), Infinite Dimensional Analysis: A Hitchhiker's Guide, Springer, p. 229, ISBN 9783540326960.

• citerefconway1990Conway, John B. (1990), "III.2 Linear Operators on Normed Spaces", A Course in Functional Analysis, New York: Springer-Verlag, pp. 67–69, ISBN 0-387-97245-5

• citerefdiestel1984Diestel, Joe (1984). Sequences and series in Banach spaces. New York: Springer-Verlag. ISBN 0-387-90859-5. OCLC 9556781.

• citerefrudin1991Rudin, Walter (1991). Functional Analysis. International Series in Pure and Applied Mathematics. Vol. 8 (Second ed.). New York, NY: McGraw-Hill Science/Engineering/Math. ISBN 978-0-07-054236-5. OCLC 21163277.